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ABSTRACT 



An independent evaluation of the College of Lake County's 
National Workplace Literacy Program included both formative and summative 
evaluations over the 3 years of the project. The evaluation design was 
planned primarily as a naturalistic inquiry that used the structured 
interview approach to data collection. An external evaluator interviewed all 
project stakeholders during scheduled visits over the 3 -year period to 
identify project strengths and recommendations for improvement. In addition, 
the evaluator visited classes at various sites and provided feedback on the 
instruction. The project had several unique aspects; development of a list of 
learning objectives and curriculum that could be used as a model across all 
sites; development of an assessment instrument for use with 
English- as -a- Second-Language students; and inservice education provided to 
part-time adult educators. The evaluator provided an introductory inservice 
session to the project staff on the Kirkpatrick evaluation model. Interview 
data were entered into a Filemaker Pro Database to detect trends in data 
across time and sites. Included in the trends were that all stakeholders were 
impressed by the changes evident in the learners and that staff viewed the 
project as a learning experience for themselves. Findings indicated the 
project was well implemented and became more central to the businesses 
concerned as time went by. (The five-page report is followed by these 
appendixes: structured interview forms, interview data, and analysis of 
trends.) (YLB) 
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EVALUATION REPORT OF THE COLLEGE OF LAKE COUNTY'S 
NATIONAL WORKPLACE LITERACY PROGRAM GRANT 



This report is an independent evaluation of the three-year project granted to the 
College of Lake County under the National Workplace Literacy Program (NWLP) 
of the U. S. Department of Education. The external evaluator, who had also 
provided an independent evaluation during the previous 18 -month grant cycle 
awarded to the College of Lake County, was engaged at the beginning of the 
project. While she did not influence the statement of the project objectives, she 
did work with the project team from the outset to develop the evaluation design. 
The involvement of the project stakeholders upfront in the evaluation design 
has been widely recommended in the evaluation research literature as the 
optimal approach to evaluation. 

The Institute for the Study of Adult Literacy at Penn State was contracted to 
provide the evaluation services. Dr. Eunice N. Askov, Institute Director and 
Professor of Education, was named the external evaluator. The contract was in 
the form of consulting days that would be devoted to the project. Two visits of 
three days duration each were planned for the first and third years. One three- 
day visit was planned for the second year of implementation. Consulting days for 
data analyses and writing were also provided for each year of the project. This 
design was implemented as planned with one exception: During the third year 
only one site visit actually occurred due to cutbacks in the grant to the College. 

At the conclusion of each visit the external evaluator wrote a letter outlining 
commendations and recommendations. These letters were included in the 
project's annual reports to the U. S. Department of Education. 

While the external evaluator was at the project site for only limited periods of 
time, she and the project team maintained periodic contact during the three-year 
period. Communication was handled throughout the project through 
telephone, fax, and email. Products and project updates were sent to the external 
evaluator periodically. Furthermore, informal visits with the project team 
occurred during various conferences, such as the Workplace Learning 
Conferences held in April in Milwaukee. She also heard presentations from the 
project team at the annual regional adult education inservice day for northern 
Illinois as well as at other national conferences. 

Evaluation Design 

The evaluation design was planned primarily as a naturalistic inquiry which 
used the structured interview approach to data collection. (Sample interview 
forms for partners, training directors/supervisors, learners, and project staff are 
provided in Appendix A.) The external evaluator interviewed all the project 
stakeholders during the scheduled visits over the three-year period to identify 
project strengths and recommendations for improvement. In addition, the 
evaluator visited classes at various sites and provided feedback on the 
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instruction that was observed. Thus, the project included both formative as well 
as summative evaluation. 

Since project objectives had been stated upfront as part of the contract of the 
College of Lake County with the US Department of Education, the 
accomplishment of these objectives was also monitored during the three-year 
project. (A detailed discussion of these are included in the Final Report of the 
College of Lake County.) In addition, the Kirkpatrick model (Kirkpatrick, D., 

1994, Evaluating Training Programs, San Francisco, CA: Barrett-Koehler) was 
used as guidance to the evaluator and the project team for summative 
evaluation. Since the evaluator was not located at the project site, and since she 
was contracted for a limited number of consulting days throughout the project, 
the data for the Kirkpatrick evaluation model were collected by the project team. 

The remainder of this report is structured according to the approaches to 
evaluation that were conducted. At the conclusion of the report a holistic 
evaluation summary is provided. 

Accomplishment of the Grant Objectives 

The Final Report of the College of Lake County to the US Department of 
Education summarizes concisely the accomplishment of the grant objectives. 

The ten project objectives are described accurately. Evidence for their 
accomplishment is objectively provided. There is no need to repeat this aspect of 
the evaluation study here since it is provided in the project's Final Report. 

Q 

The Final Report also describes the setting for the project, namely in eight 
manufacturing companies in the Lake County area. Workers were released on 
the clock or paid for the time in classes. (The money used to pay the workers to 
attend classes was counted by the companies as the required match for the grant.) 

The project was originally housed in the economic development division of the 
College since it was perceived to contribute to the economic development of 
Lake County. After about a year into the project, it was transferred to the adult 
basic skills division under different College oversight. Its offices were also 
moved from the more centrally located business complex to an outlying building 
which housed the adult basic skills programs. 

The project proactively attempted to use the data collection instrument provided 
and mandated by the US Department of Education known as NWLIS. For the 
first two years of the project the full-time assessment specialist collected and 
entered the data into the computerized database. While numerous problems 
with the data collection system existed that were beyond the control of the project 
team, the NWLIS data do profile the learners in the project. The data from that 
instrument are also reported in the Final Report. 
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The project was unique in that it aspired to more than simply providing 
workplace literacy services in eight area manufacturing companies. While it 
certainly did provide instruction that was customized to the workplace, it also 
sought to develop a list of learning objectives and curriculum that could be used 
across all sites. These are model efforts that were part of the spirit of being a 
national demonstration program. Much of the project effort throughout the 
three years was devoted to curriculum development. 

Furthermore, the project team also developed an assessment instrument for use 
with English-as-a-Second-Language (ESL) learners. They were not satisfied with 
the available commercial and informal instruments. Therefore, they developed 
and piloted an instrument that uses workplace contextual pictures to assess a 
learner's language competency. It is highly commendable that the project would 
go as far as they did with a test development effort. (This assessment instrument 
should be further developed and commercialized through a more rigorous 
validation process as well as assessed for its test reliability and predictiveness.) 

Another unique aspect of this project was the inservice education provided to 
part-time adult educators in the Lake County area. Since the College hires only 
part-time instructors for its adult education programs, one of the project goals 
was to develop a cadre of trained instructors for workplace literacy programs. 

The project effectively accomplished this goal through various inservice 
programs. The curriculum described above also provided the structure needed 
for instructors who were previously inexperienced in workplace literacy. 

Finally, the partnership of area businesses created by the project was extremely 
strong and beneficial. Companies which might otherwise even be competitors 
worked together to solve mutual problems, even those beyond workplace 
literacy. The regular partnership meetings across sites were highly commendable 
in fostering a spirit of collaboration at the local level. 

Evaluation Approach Using Kirkpatrick's Model 

The evaluator provided an introductory inservice session to the project staff on 
the Kirkpatrick model. Using a handout which described the four levels of 
evaluation— satisfaction of all stakeholders, mastery of the skills taught, transfer 
of instruction to the workplace, and impact on the organizations— the staff then 
brainstormed how the levels could be used in data collection. They decided on 
procedures for collecting the data used as evidence for the accomplishment of 
each of the four levels of evaluation. Monitoring the data collection for these 
levels of evaluation served as a focal point for the external evaluator's 
subsequent visits. One member of the project team served as the lead in this data 
collection effort. She worked closely with the project instructors as well as with 
the companies to obtain the data. The results are reported in detail in the College 
of Lake County's Final Report. 
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As one inspects the data that are reported, one can see that this was a highly 
effective project. It is also clear that the data that are reported are honest--in 
other words, no claims are made for total mastery, transfer, or impact. However, 
it is clear that over the three-year project that a very positive impact was realized 
by all stakeholders. Therefore, in terms of the Kirkpatrick evaluation hierarchy, 
the project was judged to be highly effective. 

A question must be raised about the cost effectiveness of the project. In terms of 
delivering services in the local area, it would not be considered cost effective, 
given the size and qualifications of the staff. If the project, however, is viewed in 
terms of its total intent--that of developing curriculum and building the capacity 
of the local part-time teaching cadre-then it could be as cost effective as any of 
the national demonstration projects. 

Structured Interview Data Analysis 

Qualitative analysis of the interview data collected by the external evaluator 
occurred with the assistance of a staff person at the Institute for the Study of 
Adult Literacy. The assistant entered the responses that were recorded by the 
external evaluator under each question in the structured interview forms (see 
Appendix A) into a Filemaker Pro Database. Using a database permitted the 
evaluator and the assistant to see trends in the data across time and across sites. 

Some of the interviews were conducted in small groups, especially of learners. 
Others, such as with partners, were conducted individually. The project director 
and staff were not in attendance for most of the interviews with company 
stakeholders. These interview reports are provided in Appendix B. (The names 
of the individuals and companies are removed to protect confidentiality that was 
assured during the interviews.) 

The analysis of trends is provided in Appendix C. It is apparent that all 
stakeholders were impressed by the changes that were evident in the learners. 
Often this anecdotal evidence convinced them of the worthwhile nature of the 
project. Learners also saw changes in themselves. They felt increased confidence 
and self-esteem. Many began other learning projects outside the company, for 
example, by taking classes at the community college. Perhaps the most 
noteworthy of the comments made by the supervisors was that the workers 
became more independent. While scheduling the release of workers on the 
clock was difficult, it was perceived to be worth the effort because supervisors did 
not have to spend time reading and reviewing instructions for the workers. 

Furthermore, the staff viewed the project as a tremendous learning experience 
for themselves. The project director pushed all of them to become involved in 
the dissemination of the project which brought about professional growth 
among the staff. Some of the business partners also became actively involved in 
project dissemination which opened up new opportunities for them. 



Evaluation Summary 

The NWLP project at the College of Lake County was well implemented. In fact, 
as time went by, the project became more central to the businesses concerned. 
Companies that had already made a commitment to workforce education at the 
beginning of the project grew in their commitment and sophistication of 
implementation. The leadership of the project staff and the curriculum 
materials produced were highly regarded. Most of the companies wanted to 
institutionalize the program at the end of the three years. All wanted to 
continue the relationship with the College of Lake County. 

Not surprisingly, program institutionalization often seemed to depend on the 
leadership of key business partners. In one small company, for example, the 
business partner remained constant throughout the three-year period. He 
became so involved in the project that he collected his own data that provided 
evidence for the effectiveness of the project. He, in fact, became an outstanding 
business spokesman in the project dissemination. On the other hand, where 
leadership changed during the period, or where other factors intervened (such as 
lack of consistent support from the parent company or economic downturns), 
institutionalization became problematic. 

Unfortunately, the project always seemed to have a marginalized status in the 
College. Although the immediate supervisor of the project recognized the 
merits of continuing the project, others did not seem to recognize its value. The 
ambitious project was not deemed to be cost effective without governmental 
support. Rather than using the cadre of talent that had been developed by the 
project to extend its reach into area businesses, the College chose not to continue 
the project. Perhaps its location in the adult basic skills program worked against 
this type of entrepreneurial activity which regularly occurred in the economic 
development department. 

Some state or regional structure seems to be necessary to enable projects like this 
one to continue beyond the life of the grant. Even though the state of Illinois 
offers workplace literacy grants, it appears that an organizing state structure is 
more likely to assure continuation of a demonstration project. It seemed to be 
difficult for the College, on its own, to move from a government-supported 
national demonstration project to locally delivered services supported by area 
businesses. Perhaps because the project was more ambitious than just delivering 
services to area businesses, it was perceived to be beyond the scope of the local 
college. For this reason this evaluator recommends that future national 
demonstration projects be channeled through the states to encourage the 
development of state support structures to ensure institutionalization of 
innovation. 
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APPENDIX A 



Structured Interview Forms 



COLLEGE OF LAKE COUNTY PROJECT 
Workplace Literacy Project 
Learner Interview Guide 
April 15-17, 1997 

1 . Place of Employment: ^ 

2. Name of Class: ^ 

3. How satisfied were you with the class? Why? 



4. What was the most important part? Least important? 



5 . What did you gain from the class? 

6. How did the class help you with your job? Examples? 

Reading? 

Writing? 

Speaking? 

Listening? 

Math? 

Teamwork? 

7. Did the class help you understand the company better? Examples? 

8. Do you feel better about yourself as a worker as a result of the class? 



9. Did the class prepare you for a company training program? Which one? 




10. Did the class help you with getting a promotion or a better job? How? 

1 1 . How did your fellow workers feel about you taking the class? 

1 2. Would you recommend others to take the class? 

13. Did you get support from your supervisor to attend the class? 

14. Do you look forward to any more classes? Where? 

15. Do you do any more reading, writing, or math at work than you did before the class? Examples? 

1 6. Do you do any more reading, writing, or math at home than you did before the class? Examples? 

17. How did the class help you outside the job? Examples? 

Family? 

Community? 

Voting? 



18. Other comments: 



COLLEGE OF LAKE COUNTY PROJECT 
Workplace Literacy Project 
Interview Guide for Partners 
April 28, 1997 



1 . Place of employment 

2. How satisfied are you with the project? Why? 



3. How effective was the partnership between industry and the College? 



4. Did your expectations change during the course of the project? How? 



5 . What were your major disappointments? 



6. How did the company benefit (productivity, quality, safety, absenteeism, retention, etc.)? Examples? 



7. How did the workers benefit (morale, attendance, teamwork, etc.)? Examples? 



8 . How cost-effective was the project? 



9 . How do you feel about continuing the project? 



10. Has the project helped the company with public relations (newspaper articles, TV, or radio coverage, 
etc.)? Examples? 



11. Has the project improved the company’s training program? Examples? 



12. What changes do you see in the near future that would change the needs of your workers for training 



13. Would you recommend this training program to your colleagues in other companies? 



14. Other comments: 



COLLEGE OF LAKE COUNTY PROJECT 
Workplace Literacy Project 
Supervisor/Training Director Interview Guide 
April 15-17, 1997 

1 . Place of Employment: 

2. Name of Class: 

3 . Number of your workers who participated: ^ 

4. How satisfied were you with the class(es)? Why? 



5. How did the company benefit (productivity, quality, safety, absenteeism, retention, etc.)? Examples? 



6. How did the workers benefit (morale, attendance, teamwork, etc.)? Examples? 



7 . Has participation in the class(es) affected their chances for advancement? 



8. How much did the workers talk to you about the class(es)? 



O 
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9 . How did the workers who participated feel about the class(es)? 



10. How did the other workers feel about the class(es)? 



1 1 . How did you feel about releasing workers from the job? How did you accommodate? 



12. How does this training compare with training the company has done or could do itself? 



13. Would you recommend the company continue this kind of training? 



14. What are the advantages and disadvantages of working with the College in offering the class(es)? 



15. Other comments: 



COLLEGE OF LAKE COUNTY PROJECT 
Workplace Literacy Project 
Staff Interview Guide 
April 15-17, 1997 

1 . How satisfied are you with the project? 



2. What are the greatest satisfactions? 



3. To what extent are there agreements on the goals among all stakeholders? 



4. What factors helped with the success of the project? 



5 . What factors acted as deterrents to the project? 



6. What do you see as the major outcomes? 



7 . What are the major disappointments? 





8 . What was the most difficult part of the project? 



9. How do you feel about your linkage with industry? Will it continue? 



10. What would you change in a future project? 



1 1 . How has the college benefited from the project? 



12. How much support have you had from the college? 



13. How cost-effective was the project? 



14. What are your plans for the future regarding this program? 



15. Other comments: 



APPENDIX B 



Interview Data 



Supervisor/Training Director Interview Guide 
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How did the workers lit is easier to communicate, both inside and outside. It is also easier for employees to 

benefit (morale, atten- move Up. 

dance, teamwork, etc.)? 

Examples? 
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What factors acteci as IThese factors were mentioned: time, incentives for parttcipants (varial 
(deterrents to the project? companies— teachers need to know what to do). 
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18. other comments: Reading/writing classes should be offered if at appropriate level. 

Learners 8, 9, 10 and 1 1 
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18. other comments: This learner wants more job-related reading and writing. One of the highest in the 

class-wants a higher level class. Learner 1 
August 14-16, 1995 
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Summary of Trends — Trainers 



Trainers at these companies seemed to be impressed by the effects of the program 
on students. Most noted that students showed increased self-esteem, 
motivation, and trainability. Communications had also improved. It was noted 
that this program had the effect of showcasing the needs for continual training in 
these companies. 

Question 1: Place of Employment? 

The five respondents were employed at Stone (2), Baxter, General Metal, and 
Reliable. 

Question 2: Name of Class? 

Three respondents represented ESL classes; the other two math classes. 

Question 3: Number of workers who participated? 

Answers to this question varied widely, from "all" to "5 of 13" and "6 of 14." 
These responses could cause the observer to conclude that the interviewees 
did not understand the question. 

Question 4: How satisfied were you with the class(es)? Why? 

The interviewees were generally satisfied or very satisfied with the classes. 
They noted that employee performance has improved and self-confidence has 
increased. Workers now understand product orders and workplace materials 
better, and they are more eager to obtain additional training. 

Question 5: How did the company benefit (productivity, quality, safety, 
absenteeism, retention, etc.)? Examples? 

One respondent stated that it was now easier to deal with the employees on 
both personal and professional levels, since workers are now able to work 
more independently. Less mistakes are being made on the job, said another 
respondent. Intangible benefits, such as improved morale and self-confidence, 
were also noted. 

Question 6: How did the workers benefit (morale, attendance, teamwork, etc.)? 
Examples? 

It is easier for the workers to communicate with supervisors, management, 
and each other. Some employees have gotten GEDs and are now pursuing 
additional educational and employment opportunities. One respondent 



mentioned that an ESL student has begun reading the English language 
newspaper every day, as a result of improved reading skills. 

Question 7: Has participation in the class(es) affected their chances for 
promotion? 

Two respondents mentioned that since their companies now required workers 
to have GEDs or equivalent high-school achievement in order to advance, 
participation in the literacy program will eventually positively affect chances 
for promotion. One interviewee mentioned an incident where workers, 
newly confident in their English-speaking ability, questioned a supervisor 
who had given them a work directive. It turned out that the workers were 
correct to question the supervisor, who admitted his mistake. 

Question 8: How much did the workers talk to you about the class(es)? 

Only one of the respondents stated that he talked to workers frequently about 
the classes. However, he said that he received a great deal of feedback about 
the positive relationship of the course work to the job. Another interviewee 
said that students come to him when they have questions or problems. The 
other three respondents noted that they do not have much interaction with 
the workers on this topic. 

Question 9: How do the workers who participated feel about the class(es)? 

All of the respondents stated that the workers felt good, enthusiastic, and 
better about themselves, as a result of the class. One interviewee mentioned 
that students "clean up" for class. Another respondent said that students had 
been saying that classes should have been offered long ago. 

Question 10: How do the other workers feel about the classes? 

Some other workers resent that they have to cover for those attending class, 
said one respondent. Another company has a waiting list for classes — many 
want to participate, but there is not adequate space. One interviewee noted 
that at his company some workers were resentful because this program 
addressed only basic skills, not "higher ESL." One training director 
complained that most of the workers in his company were "complacent" and 
did not prioritize education. 

Question 11: How do you feel about releasing workers from the job? How did 
you accommodate? 

Most of the respondents said that some schema had been devised in order to 
allow workers to attend classes. However, companies with small departments 
found it difficult to cover for workers who were not on the shop floor. One 



interviewee mentioned that his company "works through the problems" that 
surround this issue because they "understand the benefits" of a literacy 
program for workers. 

Question 12: How does this training compare with training the company has 
done or could do itself? 

Only one respondent mentioned that his company offers a wide range of safety 
and other types of training. Two of the companies offer "on the job" training 
only, while another offers only technical training. One noted that his 
company was just beginning a formal training program. 

Question 13: Would you recommend the company continue this type of 
training? 

All respondents gave emphatic positive responses to this question. Words 
like "Absolutely!" "100%" and "strongly encourage" were used. 

Question 14: What are the advantages and disadvantages of working with the 
College in offering these class(es)? 

One interviewee said that although the company had more specialized 
knowledge, the college has more expertise in learning concepts, this 
insinuating that the partnership was a positive one. "The college is a 
resource," said one respondent, who appreciated that the College had helped 
to create a curriculum especially for his employees. Some workers now take 
classes at the College — the partnership encouraged them to pursue additional 
education. Another respondent said that the College was very helpful and 
methodical about helping to set up the program. However, the same 
respondent stated that he would like teachers hired by the College to be more 
flexible, especially in terms of accommodating workers who might have to 
work late, etc. 

Question 15: Other comments? 

One trainer noted that advanced ESL classes are needed at his company and 
added that his involvement in this program had inspired him to pursue 
advanced education on his own. Some of the hourly English-speaking 
workers want to learn Spanish so that they can communicate with co-workers. 
One person noted that worker complacency is the biggest problem at his 
company but that the complacency would soon be shaken loose as new 
machinery would soon start to replace workers. 
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Summary of Trend — Staff 



Staff members, overall, reported varying degrees of satisfaction with the 
program. Early interviews suggested that there had been problems with program 
implementation, with receiving funding from the funder, and with getting 
support from the college. However, later interviews revealed that many of these 
bugs had been worked out and that the program began to run smoothly in each 
company. Staff members expressed concern over the continuation of the 
program once the funding had ceased; many were not optimistic that the 
program would continue solely under the aegis of the companies involved. 

Question 1: How satisfied are you with the project? 

Three of the interviewees said they were "very satisfied" or "satisfied" with 
the project; one said it was too soon to tell. One enthusiastic respondent said 
she would give the project a "4 on a 5 point scale." No interviewee said he or 
she was dissatisfied with the project. 

Question 2: What are the greatest satisfactions? 

Satisfaction levels were primarily affected by the progress of students who 
were participating, the development of workers' thinking skills, the materials 
being used, the staff development, and the cooperation of the partners. One 
respondent mentioned that she enjoyed using photography with her class. 

Question 3: To what extent are there agreements on the goals among all 
stakeholders? 

A high degree of cooperation among the stakeholders was cited. Most said 
that cooperation had been "good." One interviewee was not certain of the 
management buy-in at her company, however. Stakeholders, according to the 
respondents, that uncovering students' needs and specializing the curricula to 
meet those needs were of importance. 

Question 4: What factors helped the success of the project? 

The linkage with the College, the teaching staff, the materials, the enthusiasm 
and interaction of the learners, the "cooperative spirit of the company," and 
the cooperation of the stakeholders were all mentioned as positive factors. 

Question 5: What factors acted as deterrents to the project? 

The amount of time spent in class by learners was mentioned as a deterrent; 
the interviewee felt that there should be more time allotted for classes. Multi- 
leveling in classes (the practice of keeping students of all achievement levels 
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in the same class) was seen as a deterrent, in that students could not get the 
amount of individualized and small group attention they need. This was seen 
as a classroom atmosphere that was difficult for all but the most experienced 
teachers to handle. In addition, the fact that there were not tangible incentives 
for students was seen as a deterrent. One respondent mentioned no 
deterrents. 

Question 6: What do you see as the major outcomes? 

One interviewee said that the program had affected him personally; he was 
going to pursue additional education and a transfer to another job. This 
person wants to work on developing thinking skills and problem solving 
abilities with students in the workplace setting. Another respondent 
mentioned that these workplace classes "whet the appetite" for education; 
therefore, classes should be continued and institutionalized. Improving 
relationships with partners, and the quality of the curriculum were also seen 
as major outcomes. 

Question 7: What are the major disappointments? 

One respondent stated that the program at her company was being moved to a 
low-profile location, and that it was being perceived as a "lack of esteem" for 
the program. This person also noted that adult education was harder than she 
thought it would be; although it is important, it is not valued by others. 
Recruitment, placement and retention of students was mentioned; as was the 
difficulty in finding qualified tutors and teachers who are willing to work so 
few hours per week. The gap between the end of the last grant and the 
beginning of this one (six months) was mentioned. One interviewee did not 
respond to the question. 

Question 8: What was the most difficult part of the project? 

One respondent commented on the lack of materials and the poorly ventilated 
space in which she had to run the program. There was no A-V equipment, 
only a small chalkboard and a dry mount board are available. Another teacher 
had difficulty creating and meeting objectives for a multi-level class. Time to 
plan and implement ideas was mentioned. One interviewee did not respond 
to the question. 

Question 9: How do you feel about your linkage with industry? Will it 
continue? 

Almost all the interviewees said that this linkage was "excellent" or "very 
good, " without much further comment. One interviewee did not respond to 
the question. 
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Question 10: What would you change in a future project? 

This question elicited varying responses from the interviewees. One said that 
the teacher should be more involved up-front in the creation and planning of 
the program and curricula. Another respondent would split students into 
groups according to level, would incorporate the use of computers, and would 
offer career counseling to students. A third respondent said that the program 
should be more attuned to business trends, and that an additional buy-in from 
the partners could accomplish this. Business people could also be brought in 
as guest speakers. One interviewee did not respond to the question. 

Question 11: How has the College benefited from the project? 

One respondent noted that the College "hasn't benefited as much as it could" 
from the project, saying, "They don't realize what they've got." This person 
stated that additional media coverage might be a way to improve this 
situation. Another person noted that the although the college publicized the 
program in its own PR, it could be more supportive than it has been. Other 
interviewees mentioned that workers were increasingly enrolling in courses 
at the College, in order to reskill. One person did not respond to the question. 

Question 12: How much support have you had from the College? 

Some staff development, materials, and other support have been received 
from the College, said most of the respondents. One interviewee was very 
positive about the College's support. One person mentioned that at first the 
College had been supportive, but after a while, they "stopped checking" on 
her. 

Question 13: How cost-effective was the project? 

Two respondents did not answer this question, while a third was uncertain. 
Only one said that the project was "worthwhile." 

Question 14: What are your plans for the future regarding the project? 

Two respondents did not answer this question. Another would like to do an 
inservice project for the company, while a fourth lamented that she would 
like to teach more, but cannot, since the contract will not pay for her to teach 
full time. 

Question 15: Other comments? 

Interview participants: staff working at Reliable, General Metal, and two that 
were not clearly identified. 
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Summary of Trends — Partners 

The partners seemed satisfied with the project, especially with the flexibility of the 
curriculum and with the networking possibilities between the companies and the 
College. All of the partners seemed convinced as to the worth of the program; this 
was especially apparent in later interviews in which the partners reflected on the 
return on the investment they had made in the program. They noted the positive 
effects of the program on worker retention, training, and motivation. 

Question 1: Place of Employment 

The following companies participated in this project: Reliable Power (MacLean 
Power), General Metals, Nichols Aluminum, Baxter, and Stone. 

Question 2: How satisfied were you with the project? Why? 

All the interviewees expressed satisfaction with the project, although only one 
stated that she/he was "very satisfied." Reasons for satisfaction included: 
customized and individualized curricula, staff development and networking 
among partners, numbers of learners involved, and demand for the program. 
One interviewee stated that she felt the program was flexible enough to meet the 
needs of all the company's employees. 

Question 3: How effective was the partnership between industry and the College? 

The partnership was termed "very effective," "comfortable," "flexible" and 
"cooperative." All interviewees noted that the College had been helpful in 
providing resources, materials, and other assistance to the programs. One person 
mentioned that the College had sent him to seminars for advanced training. 
Another interviewee said that the College was an active participant in the 
steering committee, and that the College was in tune with his company's needs. 
One other respondent stated that the partnership had resulted in contractual 
work with the College. 

Question 4: Did your expectations change during the course of the project? How? 

Reactions to this question were mixed. All respondents reported a change in 
expectations; however, while some changed for the positive, others were 
negative. Two respondents mentioned that they were clearer about the purpose 
of the grant than they had been at its inception. One person said that he had 
expected a shorter time frame in which to begin the project, and that he had not 
understood that the program was to be workplace-oriented until he saw the 
classes in action. Another interviewee mentioned that because the current 
program had been successful in her company, expectations for additional 
education had risen to include communications and problem-solving courses. 
One respondent stated that he had come to realize that "there is no quick fix" for 
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the literacy problems in his company. A respondent said that he had become 
much more involved in the program than he had anticipated. 

Question 5: What were your major disappointments? 

Several respondents mentioned the long time gap between the announcement of 
the program and its actual start. One also mentioned the time gap between the. 
completion of one course (ESL 1) and the start of another (ESL 2) as being de- 
motivating for the students involved. The program loses momentum unless 
classes are continuously held. Recruiting students to participate in the program 
was a problem for two of the respondents; one said that the student assessment 
and teacher-hiring processes took much too long to complete. One person stated 
that hiring a quality teacher who would teach on a very limited part-time basis 
was difficult. One interviewee also said that he had been disappointed in having 
to re-sell the program to management every year, despite its obvious successes. 

Question 6: How did the company benefit (productivity, quality, safety, 
absenteeism, retention, etc.)? Examples? 

Absenteeism, tardiness and accidents have decreased, while output, quality and 
communications have improved. One company, which works in quality teams, 
recorded that workers who were involved in the project seemed to have 
increased self confidence. ROl has increased overall for one of the companies. 
Three respondents said that it was too early for them to have noted any trends or 
positive results from the program. 

Question 7: How did the workers benefit (morale, attendance, teamwork, etc.)? 

All respondents said that the program has resulted in an increase in morale, 
enthusiasm, and self-esteem on the part of the students. An interviewee 
mentioned that it was a "status symbol" to be involved with these classes at his 
company. Another company, which improved its team system after input from 
workers involved in this project, won a state award and was invited to a large 
presentation. At another firm, one older worker was able to respond with some 
degree of eloquence to the company CEO during a ceremony. 

Question 8: How cost effective was the program? 

Responses to this question were mixed. Several respondents believed that the 
project was cost-effective, ("absolutely so," according to one respondent) 
although one had not yet figured the return on investment. One interviewee 
stated that the program was expensive, and that start-up costs were high; he is 
tracking the effect of the program as "preventative maintenance." One person 
mentioned that without the government funding, the program would not be 
happening at his company. Another said that the return on investment would be 
hard to calculate, as the program has an effect not only on skills, but on morale. 
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company loyalty, and self-confidence. One person mentioned a dramatic decrease 
(about 33%) in training time because of the program, resulting in cost-savings for 
the company. 

Question 9: How do you feel about continuing the project? 

All the respondents wish to continue the project. Some feel confident that they 
can find the finding to independently run the programs; others believe that their 
companies will not pay for the program to the same extent as was done by the 
grant. One interviewee said that he envisioned the program continuing, but on a 
smaller scale. Getting buy-ins from management, and presenting the success of 
the program as being integral to the company's success, seemed to be key issues. 

Question 10: Has the project helped the company with public relations (newspaper 
articles, TV, radio coverage, etc.)? Examples? 

Only one respondent mentioned widespread publicity of the project within the 
various branches of his corporation. Another three said that articles about the 
project had appeared in the company newsletters or in-house magazines. 

Question 11: Has the project improved the company's training program? Examples? 

All of the respondents, to one degree or another, stated that the project has made 
employees more "trainable." The learning of basic math, for instance, helps an 
employee to grasp blueprint reading concepts. Computer training and problem- 
solving courses are being asked for by the employees. One interviewee said that 
this project had the effect of creating greater demand for all sorts of in-house 
training programs. Another person said that the company had finally begun to 
see the value of training as a result of this program. 

Question 12: What changes do you see in the future that would change the needs 
of your workers for training? 

Growth of the companies, especially in the area of the addition of computers and 
automation, was seen as the greatest need for employee training. Several 
respondents also mentioned the need for workers to learn team techniques and to 
hone their problem-solving abilities, as workers are going to be called on to 
become more flexible in the future. Communications issues, such as bridging the 
gap between workers who speak different languages, is seen as a major issue. 

Question 13: Would you recommend this training program to your colleagues in 
other companies? 

All of the interviewees responded positively to this question. One person 
believes that other companies, like his own, ignored the educational needs of the 



workers in the past, but are more cognizant of them now. Another company is 
attempting to duplicate the program at another of its sites. 

Question 14: Other comments? 

One interviewee mentioned that "business is bad" at his company, thus throwing 
the continuation of the literacy program into question. Another respondent 
offered the suggestion that literacy training be offered across companies. 
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Summary of Trends — Learners 



Generally, the learners seemed to have positive responses to the program. 
Although most learners said that they were satisfied with the program, not all of 
them seemed convinced of its value to them as employees and as citizens, 
especially those who were interviewed during the early evaluations. However, 
the positive results of the program became apparent, as employees became more 
amenable to learning, more promotable, and more able to communicate with 
their co-workers and with supervisors. 

Question 1: Place of Employment? 

Learners from the following companies were interviewed: General Metal, 
Baxter, Nichols, and Stone. 

Question 2: Name of class? 

Three respondents are involved in ESL classes; one in a math class. 

Question 3: How satisfied were you with the class(es)? Why? 

Most of the learners said that satisfaction was "good" or "very good." One said 
he felt challenged; another liked the small class size. Some said that they had 
gained confidence as a result of the classes. 

Question 4: What was the most important part? Least? 

Important to the learners were the instructor, the English speaking practice, 
the learning of the metric system, and the relevance of the curriculum . One 
learner earned a GED. No least important factors were cited. 

Question 5: What did you gain from the class? 

A better knowledge of spoken and written English was most often cited as a 
gain. One respondent mentioned the knowledge of the metric system, as well 
as improvement in reasoning and math computation. 

Question 6: How did the class help you with your job? 

Students mentioned that they were more comfortable with speaking, listening 
and teamwork. One respondent mentioned that the class really did not help 
with reading and writing. However, others mentioned that they were now 
able to read specs, job orders, signs, job descriptions, and safety procedures with 
greater ease and accuracy. Two respondents said that they were also able to 
speak to supervisors now, where they could not in the past. 
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Question 7: Did the class help you understand the company better? Examples? 

Three interviewees said no, this class did not help them to understand the 
company better, while one said yes, it did. One interviewee did not respond, 
and no illustrative examples were given by any of the respondents. 

Question 8: Do you feel better about yourself as a worker as a result of the class? 

All respondents answered this question affirmatively. Self esteem has 
improved, and there is an increased feeling of confidence because of improved 
speaking and reading skills. One student said, "Having a high (not cocky) 
opinion of yourself helps." 

Question 9: Did the class prepare you for a training program? Which one? 

One student thought he was now ready for a computer class as a result of his 
studies in the program. Another said that given the chance, he would opt to 
take additional company-sponsored training courses. Some interviewees said 
that they do not need additional training unless they go to a new job. 

Question 10: Did the class help you with getting a promotion or a better job? 
How? 

All respondents answered this question positively. They stated that the 
learning of English would help them to become more eligible for supervisory 
positions. 

Question 11: How did your other workers feel about your taking the class? 

One interviewee stated that he had gotten teased as a result of his attendance, 
namely about the folder he was carrying. In one instance, co-workers said that 
the students were too old to be in school. Other respondents received positive 
feedback from their co-workers. 

Question 12: Would you recommend others to take the class? 

All of the interviewees responded affirmatively to this question. They all said 
that they have recommended the class to friends, with varying degrees of 
response. Some of the co-workers believe that they do not need additional 
education, while others showed interest. 

Question 13: Did you get support from your supervisor to attend the class? 

Most learners said they had received support from supervisors, although one 
stated that he had gotten "support, but no encouragement." 
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Question 14: Do you look forward to any more classes? Where? 

All interviewees responded yes to this question, although most had no further 
comment. One student said that he wanted to attend ESL 2 classes, while 
another wanted to take another course in anything that was offered. 

Question 15: Do you do any more reading, writing, or math at work than you did 
before the class? Examples? 

Most of the students said yes, they were doing more reading, speaking and 
writing or math on the job. The respondent from the math class said that she 
was able to perform more mathematical calculations on the job, which helped 
her to perform her job better. One respondent did not answer the question. 

Question 16: Do you do any more reading, writing, or math at home than you 
did before the class? Examples? 

Several students mentioned that they were speaking English at home more so 
than in the past. One said that he is teaching his wife to speak English, while 
another noted that he no longer watched Spanish TV, but instead viewed the 
English-speaking programs. One learner is able to help her son with math; 
another has become an avid reader. Others said that they were able to read 
newspapers and magazines with more understanding. 

Question 17: How did the class help you outside the job? Examples? 

One student said that he had started communicating with his children in 
English. Another mentioned being able to get his GED. The Spanish speakers 
do not need to communicate in English outside of the job; they live in 
communities which are Spanish-speaking. A student from the math class is 
now able to help her son with homework. 

Question 18: Other comments? 

One learner wants more job-related reading and writing classes; he said that he 
would like to see a higher-level ESL class offered. The learners realize that 
they must learn English in order to compete for better jobs. Most of these 
learners have been in the US for many years; they have not had cause to learn 
how to speak English up until recently. Other workers said that they want to 
come to school not only for themselves, but to help others. 
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